Kobo Libra 2
text-handling in Plato
The DRM/KMS framework provides the atomic API for color management through KMS
properties represented by struct drm_property. We extended the color
management interface exposed to userspace by leveraging existing resources and
connecting them with driver-specific functions for managing modeset properties.
On the AMD DC layer, the interface with hardware color blocks is established.
The AMD DC layer contains OS-agnostic components that are shared across
different platforms, making it an invaluable resource. This layer already
implements hardware programming and resource management, simplifying the external
developer s task. While examining the DC code, we gain insights into the color
pipeline and capabilities, even without direct access to specifications.
Additionally, AMD developers provide essential support by answering queries and
reviewing our work upstream.
The primary challenge involved identifying and understanding relevant AMD DC
code to configure each color block in the color pipeline. However, the ultimate
goal was to bridge the DC color capabilities with the DRM API. For this, we
changed the AMD DM, the OS-dependent layer connecting the
DC interface to the DRM/KMS framework. We defined and managed driver-specific
color properties, facilitated the transport of user space data to the DC, and
translated DRM features and settings to the DC interface. Considerations were
also made for differences in the color pipeline based on hardware capabilities.
AMD s display driver supports the following pre-defined transfer functions (aka named fixed curves):
- OETF: the opto-electronic transfer function, which converts linear scene light into the video signal, typically within a camera.
- EOTF: electro-optical transfer function, which converts the video signal into the linear light output of the display.
- OOTF: opto-optical transfer function, which has the role of applying the rendering intent .
These capabilities vary depending on the hardware block, with some utilizing hardcoded curves and others relying on AMD s color module to construct curves from standardized coefficients. It also supports user/custom curves built from a lookup table.
- Linear/Unity: linear/identity relationship between pixel value and luminance value;
- Gamma 2.2, Gamma 2.4, Gamma 2.6: pure power functions;
- sRGB: 2.4: The piece-wise transfer function from IEC 61966-2-1:1999;
- BT.709: has a linear segment in the bottom part and then a power function with a 0.45 (~1/2.22) gamma for the rest of the range; standardized by ITU-R BT.709-6;
- PQ (Perceptual Quantizer): used for HDR display, allows luminance range capability of 0 to 10,000 nits; standardized by SMPTE ST 2084.
In the AMD Display Core Next hardware pipeline, we encounter two hardware
blocks with color capabilities: the Display Pipe and Plane (DPP) and the
Multiple Pipe/Plane Combined (MPC). The DPP handles color adjustments per plane
before blending, while the MPC engages in post-blending color adjustments.
In short, we expect DPP color capabilities to match up with DRM plane
properties, and MPC color capabilities to play nice with DRM CRTC properties.
Note: here s the catch there are some DRM CRTC color transformations that
don t have a corresponding AMD MPC color block, and vice versa. It s like a
puzzle, and we re here to solve it!
struct
dpp_color_caps
and struct
mpc_color_caps.
The AMD Steam Deck hardware provides a tangible example of these capabilities.
Therefore, we take SteamDeck/DCN301 driver as an example and look at the Color
pipeline capabilities described in the file:
driver/gpu/drm/amd/display/dcn301/dcn301_resources.c
/* Color pipeline capabilities */
dc->caps.color.dpp.dcn_arch = 1; // If it is a Display Core Next (DCN): yes. Zero means DCE.
dc->caps.color.dpp.input_lut_shared = 0;
dc->caps.color.dpp.icsc = 1; // Intput Color Space Conversion (CSC) matrix.
dc->caps.color.dpp.dgam_ram = 0; // The old degamma block for degamma curve (hardcoded and LUT). Gamma correction is the new one.
dc->caps.color.dpp.dgam_rom_caps.srgb = 1; // sRGB hardcoded curve support
dc->caps.color.dpp.dgam_rom_caps.bt2020 = 1; // BT2020 hardcoded curve support (seems not actually in use)
dc->caps.color.dpp.dgam_rom_caps.gamma2_2 = 1; // Gamma 2.2 hardcoded curve support
dc->caps.color.dpp.dgam_rom_caps.pq = 1; // PQ hardcoded curve support
dc->caps.color.dpp.dgam_rom_caps.hlg = 1; // HLG hardcoded curve support
dc->caps.color.dpp.post_csc = 1; // CSC matrix
dc->caps.color.dpp.gamma_corr = 1; // New Gamma Correction block for degamma user LUT;
dc->caps.color.dpp.dgam_rom_for_yuv = 0;
dc->caps.color.dpp.hw_3d_lut = 1; // 3D LUT support. If so, it's always preceded by a shaper curve.
dc->caps.color.dpp.ogam_ram = 1; // Blend Gamma block for custom curve just after blending
// no OGAM ROM on DCN301
dc->caps.color.dpp.ogam_rom_caps.srgb = 0;
dc->caps.color.dpp.ogam_rom_caps.bt2020 = 0;
dc->caps.color.dpp.ogam_rom_caps.gamma2_2 = 0;
dc->caps.color.dpp.ogam_rom_caps.pq = 0;
dc->caps.color.dpp.ogam_rom_caps.hlg = 0;
dc->caps.color.dpp.ocsc = 0;
dc->caps.color.mpc.gamut_remap = 1; // Post-blending CTM (pre-blending CTM is always supported)
dc->caps.color.mpc.num_3dluts = pool->base.res_cap->num_mpc_3dlut; // Post-blending 3D LUT (preceded by shaper curve)
dc->caps.color.mpc.ogam_ram = 1; // Post-blending regamma.
// No pre-defined TF supported for regamma.
dc->caps.color.mpc.ogam_rom_caps.srgb = 0;
dc->caps.color.mpc.ogam_rom_caps.bt2020 = 0;
dc->caps.color.mpc.ogam_rom_caps.gamma2_2 = 0;
dc->caps.color.mpc.ogam_rom_caps.pq = 0;
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1; // Output CSC matrix.
struct dpp_color_caps,
struct mpc_color_caps
and struct rom_curve_caps.
Now, using this guideline, we go through color capabilities of DPP and MPC blocks and talk more
about mapping driver-specific properties to corresponding color blocks.
dc->caps.color.dpp.dcn_arch
dc->caps.color.dpp.dgam_ram, dc->caps.color.dpp.dgam_rom_caps,dc->caps.color.dpp.gamma_corr
AMD Plane Degamma data is mapped to the initial stage of the DPP pipeline. It
is utilized to transition from scanout/encoded values to linear values for
arithmetic operations. Plane Degamma supports both pre-defined transfer
functions and 1D LUTs, depending on the hardware generation. DCN2 and older
families handle both types of curve in the Degamma RAM block
(dc->caps.color.dpp.dgam_ram); DCN3+ separate hardcoded curves and 1D LUT
into two block: Degamma ROM (dc->caps.color.dpp.dgam_rom_caps) and Gamma
correction block (dc->caps.color.dpp.gamma_corr), respectively.
Pre-defined transfer functions:
struct drm_color_lut elements. Setting TF = Identity/Default and LUT as
NULL means bypass.
References:
struct drm_color_ctm_3x4. Setting NULL means bypass.
References:
dc->caps.color.dpp.hw_3d_lut
The Shaper block fine-tunes color adjustments before applying the 3D LUT,
optimizing the use of the limited entries in each dimension of the 3D LUT. On
AMD hardware, a 3D LUT always means a preceding shaper 1D LUT used for
delinearizing and/or normalizing the color space before applying a 3D LUT, so
this entry on DPP color caps dc->caps.color.dpp.hw_3d_lut means support for
both shaper 1D LUT and 3D LUT.
Pre-defined transfer function enables delinearizing content with or without
shaper LUT, where AMD color module calculates the resulted shaper curve. Shaper
curves go from linear values to encoded values. If we are already in a
non-linear space and/or don t need to normalize values, we can set a Identity TF
for shaper that works similar to bypass and is also the default TF value.
Pre-defined transfer functions:
calculate_curve() function in the file
amd/display/modules/color/color_gamma.c.struct drm_color_lut elements. When setting Plane Shaper TF (!= Identity)
and LUT at the same time, the color module will combine the pre-defined TF and
the custom LUT values into the LUT that s actually programmed. Setting TF =
Identity/Default and LUT as NULL works as bypass.
References:
dc->caps.color.dpp.hw_3d_lut
The 3D LUT in the DPP block facilitates complex color transformations and
adjustments. 3D LUT is a three-dimensional array where each element is an RGB
triplet. As mentioned before, the dc->caps.color.dpp.hw_3d_lut describe if
DPP 3D LUT is supported.
The AMD driver-specific property advertise the size of a single dimension via
LUT3D_SIZE property. Plane 3D LUT is a blog property where the data is interpreted
as an array of struct drm_color_lut elements and the number of entries is
LUT3D_SIZE cubic. The array contains samples from the approximated function.
Values between samples are estimated by tetrahedral interpolation
The array is accessed with three indices, one for each input dimension (color
channel), blue being the outermost dimension, red the innermost. This
distribution is better visualized when examining the code in
[RFC PATCH 5/5] drm/amd/display: Fill 3D LUT from userspace by Alex Hung:
+ for (nib = 0; nib < 17; nib++)
+ for (nig = 0; nig < 17; nig++)
+ for (nir = 0; nir < 17; nir++)
+ ind_lut = 3 * (nib + 17*nig + 289*nir);
+
+ rgb_area[ind].red = rgb_lib[ind_lut + 0];
+ rgb_area[ind].green = rgb_lib[ind_lut + 1];
+ rgb_area[ind].blue = rgb_lib[ind_lut + 2];
+ ind++;
+
+
+
+ /* Stride and bit depth are not programmable by API yet.
+ * Therefore, only supports 17x17x17 3D LUT (12-bit).
+ */
+ lut->lut_3d.use_tetrahedral_9 = false;
+ lut->lut_3d.use_12bits = true;
+ lut->state.bits.initialized = 1;
+ __drm_3dlut_to_dc_3dlut(drm_lut, drm_lut3d_size, &lut->lut_3d,
+ lut->lut_3d.use_tetrahedral_9,
+ MAX_COLOR_3DLUT_BITDEPTH);
dc->caps.color.dpp.ogam_ram
The Blend/Out Gamma block applies the final touch-up before blending, allowing
users to linearize content after 3D LUT and just before the blending. It supports both 1D LUT
and pre-defined TF. We can see Shaper and Blend LUTs as 1D LUTs that are
sandwich the 3D LUT. So, if we don t need 3D LUT transformations, we may want
to only use Degamma block to linearize and skip Shaper, 3D LUT and Blend.
Pre-defined transfer function:
struct drm_color_lut elements. If plane_blend_tf_property != Identity TF,
AMD color module will combine the user LUT values with pre-defined TF into the
LUT parameters to be programmed. Setting TF = Identity/Default and LUT to NULL
means bypass.
References:
struct drm_color_lut elements. Setting NULL means bypass.
Not really supported. The driver is currently reusing the DPP degamma LUT block
(dc->caps.color.dpp.dgam_ram and dc->caps.color.dpp.gamma_corr) for
supporting DRM CRTC Degamma LUT, as explaning by [PATCH v3 20/32]
drm/amd/display: reject atomic commit if setting both plane and CRTC
degamma.
dc->caps.color.mpc.gamut_remap
It sets the current transformation matrix (CTM) apply to pixel data after the
lookup through the degamma LUT and before the lookup through the gamma LUT. The
data is interpreted as a struct drm_color_ctm. Setting NULL means bypass.
dc->caps.color.mpc.ogam_ram
After all that, you might still want to convert the content to wire encoding.
No worries, in addition to DRM CRTC 1D LUT, we ve got a AMD CRTC gamma transfer
function (TF) to make it happen. Possible TF values are defined by enum
amdgpu_transfer_function.
Pre-defined transfer functions:
struct drm_color_lut elements. When setting CRTC Gamma TF (!= Identity)
and LUT at the same time, the color module will combine the pre-defined TF and
the custom LUT values into the LUT that s actually programmed. Setting TF =
Identity/Default and LUT to NULL means bypass.
References:
color_range and color_encoding properties. It is used for color space
conversion of the input content. On the other hand, we have de DC Output CSC
(OCSC) sets pre-defined coefficients from DRM connector colorspace
properties. It is uses for color space conversion of the composed image to the
one supported by the sink.
References:
armhf and arm64) is not even
mentioned. Is this really true? It is, after all a Debian-based
system, so it should in theory be doable. Let s try!
We have always said that for a major version upgrade, you should re-image your SD card and start again with a clean image. In the past, we have suggested procedures for updating an existing image to the new version, but always with the caveat that we do not recommend it, and you do this at your own risk. This time, because the changes to the underlying architecture are so significant, we are not suggesting any procedure for upgrading a Bullseye image to Bookworm; any attempt to do this will almost certainly end up with a non-booting desktop and data loss. The only way to get Bookworm is either to create an SD card using Raspberry Pi Imager, or to download and flash a Bookworm image from here with your tool of choice.Which means, it s time to actually try it turns out it s actually trivial, if you use RPIs as headless servers. I had only three issues:
initrd, the new initrd-building scripts/hooks are
looking for some binaries in /usr/bin, and not in /bin;
solution: install manually the usrmerge package, and then re-run
dpkg --configure -a;/boot/config-$(uname -r), and the raspberry pi
kernel package doesn t provide this; workaround: modprobe configs && zcat /proc/config.gz > /boot/config-$(uname -r);/etc/network/interface, migrating
from the previous dhcpcd to NetworkManager will break network
connectivity, and require you to log in locally and fix things.login.defs
being rewritten from scratch and showing a baffling diff, for
example), it was easy.
Is it worth doing? Definitely. Had no data loss, and no non-booting
system.
devscripts, since you will need dget dpkg --add-architecture arm64whiptail:amd64 and rejoice at
running a 64-bit binary!dget
and dpkg -iarmhf and arm64 versions
before doing dpkg -i, since you ll need to rollback some
installsdpkg and apt to
arm64, at which point the default architecture flips over; from
here, if you ve done it at the right moment, it becomes very easy;
you ll probably need an apt install --fix-broken, though, at
firstarm64 versionsdpkg --remove-architecture armhf, reboot, and profit!apt-show-versions to see which packages have
versions that are no longer in any repo, then downgrade them.
There s a further, minor, complication that there were about 3-4
packages with same version but different hash (!), which simply needed
apt install --reinstall, I think.
arch:any), that is
fine. But architecture-independent packages (arch:all) are
problematic since usually they depend on a binary package, but they
always depend on the default architecture version!
Hrmm, and I just realise I don t have logs from this, so I m only ~80% confident. But basically:
vim-solarized (arch:all) depends on vim (arch:any)vim-solarized, until the default architecture becomes arm64Multi-Arch: foreign solves this, per the debian wiki which says:
Note that even thoughIt also has warnings about how to properly use this. But, in general, not many packages have it, so it is a problem.Architecture: allandMulti-Arch: foreignmay look like similar concepts, they are not. The former means that the same binary package can be installed on different architectures. Yet, after installation such packages are treated as if they were native architecture (by definition the architecture of the dpkg package) packages. Thus Architecture: all packages cannot satisfy dependencies from other architectures without being marked Multi-Arch foreign.
dpkg -i)# apt install systemd:arm64 --no-install-recommends
The following packages will be REMOVED:
systemd
The following NEW packages will be installed:
systemd:arm64
0 upgraded, 1 newly installed, 1 to remove and 35 not upgraded.
Do you want to continue? [Y/n] y
dpkg: systemd: dependency problems, but removing anyway as you requested:
systemd-sysv depends on systemd.
Removing systemd (247.3-7+deb11u2) ...
systemd is the active init system, please switch to another before removing systemd.
dpkg: error processing package systemd (--remove):
installed systemd package pre-removal script subprocess returned error exit status 1
dpkg: too many errors, stopping
Errors were encountered while processing:
systemd
Processing was halted because there were too many errors.dpkg -i
from /var/cache/apt/archives.
In this case it manifested via a prerm script, in other cases is
manifests via dependencies that are no longer satisfied for packages
that can t be removed, etc. etc. So you will have to resort to dpkg -i a lot.
lib package,
in one go, since these can coexist.
Well, this simple plan is complicated by the fact that some packages
are named libfoo-bar, but are actual holding (e.g.) the bar binary
for the libfoo package. Examples:
libmagic-mgc contains /usr/lib/file/magic.mgc, which conflicts
between the 32 and 64 bit versions; of course, it s the exact same
file, so this should be an arch:all package, but libpam-modules-bin and liblockfile-bin actually contain binaries
(per the -bin suffix)# apt install $(dpkg -i grep ^ii awk ' print $2 ' grep :amrhf sed -e 's/:armhf/:arm64')
xzcat haos_generic-x86-64-11.0.img.xz dd of=/dev/mmcblk0 bs=1M
That just worked, prefectly and really fast. If you want to use a GUI in a live environment, then just using
the gnome-disk-utility ("Disks" in Gnome menu) and using the "Restore Disk Image ..." on a partition would
work just as well. It even supports decompressing the XZ images directly while writing.
But that image is small, will it not have a ton of unused disk space behind the fixed install partition? Yes,
it will ... until first boot. The HA OS takes over the empty space after its install partition on the first
boot-up and just grows its main partition to take up all the remaining space. Smart. After first boot is
completed, the first boot wizard can be accessed via your web browser and one of the prominent buttons there
is restoring from backup. So you just give it the backup file and wait. Sadly the restore does not actually give
any kind of progress, so your only way to figure out when it is done is opening the same web adress in another
browser tab and refresh periodically - after restoring from backup it just boots into the same config at it had
before - all the settings, all the devices, all the history is preserved. Even authentification tokens are
preserved so if yu had a Home Assitant Mobile installed on your phone (both for remote access and to send
location info and phone state, like charging, to HA to trigger automations) then it will just suddenly start
working again without further actions needed from your side. That is an almost perfect backup/restore experience.
The first thing you get for using the OS version of HA is easy automatic update that also automatically takes
a backup before upgrade, so if anything breaks you can roll back with one click. There is also a command-line
tool that allows to upgrade, but also downgrade ha-core and other modules. I had to use it today as HA version
23.10.4 actually broke support for the Sonoff bridge that I am using to control Zigbee devices, which are like
90% of all smart devices in my home. Really helpful stuff, but not a must have.
What is a must have and that you can (really) only get with Home Assistant Operating System are Addons. Some
addons are just normal servers you can run alongside HA on the same HA OS server, like MariaDB or Plex or a
file server. That is not the most important bit, but even there the software comes pre-configured to use in
a home server configuration and has a very simple config UI to pre-configure key settings, like users,
passwords and database accesses for MariaDB - you can litereally in a few clicks and few strings make serveral
users each with its own access to its own database. Couple more clicks and the DB is running and will be kept
restarted in case of failures.
But the real gems in the Home Assistant Addon Store are modules that extend Home Assitant core functionality
in way that would be really hard or near impossible to configure in Home Assitant Container manually,
especially because no documentation has ever existed for such manual config - everyone just tells you to
install the addon from HA Addon store or from HACS. Or you can read the addon metadata in various repos and
figure out what containers it actually runs with what settings and configs and what hooks it puts into the HA
Core to make them cooperate. And then do it all over again when a new version breaks everything 6 months later
when you have already forgotten everything. In the Addons that show up immediately after installation are
addons to install the new Matter
server, a MariaDB and MQTT server
(that other addons can use for data storage
and message exchange), Z-Wave support and ESPHome integration and very handy File manager that includes
editors to edit Home Assitant configs directly in brower and SSH/Terminal addon that boht allows SSH
connection and also a
web based terminal that gives access to the OS itself and also to a comand line interface, for example, to
do package downgrades if needed or see detailed logs. And also there is where you can get the features
that are the focus this year for HA developers - voice enablers.
However that is only a beginning. Like in Debian you can add additional repositories to expand your list of
available addons. Unlike Debian most of the amazing software that is available for Home Assistant is outside
the main, official addon store. For now I have added the most popular addon repository - HACS (Home Assistant
Community Store) and repository maintained by Alexbelgium. The first includes things like NodeRED (a workflow
based automation programming UI), Tailscale/Wirescale for VPN servers, motionEye for CCTV control, Plex for
home streaming. HACS also includes a lot of HA UI enhacement modules, like themes, custom UI control panels
like Mushroom or mini-graph-card and integrations that provide more advanced functions, but also require
more knowledge to use, like Local Tuya - that is harder to set up, but allows fully local control of (normally)
cloud-based devices. And it has AppDaemon - basically a Python based automation framework where you put in
Python scrips that get run in a special environment where they get fed events from Home Assistant and can
trigger back events that can control everything HA can and also do anything Python can do. This I will need
to explore later.
And the repository by Alex includes the thing that is actually the focus of this blog post (I know :D) -
Firefly III addon and Firefly Importer addon that you can then add to your Home Assistant OS with a few
clicks. It also has all kinds of addons for NAS management, photo/video server, book servers and
Portainer that
lets us setup and run any Docker container inside the HA OS structure. HA OS will detect this and warn you
about unsupported processes running on your HA OS instance (nice security feature!), but you can just dismiss
that. This will be very helpful very soon.
This whole environment of OS and containers and apps really made me think - what was missing in Debian that
made the talented developers behind all of that to spend the immense time and effor to setup a completely
new OS and app infrastructure and develop a completel paraller developer community for Home Assistant apps,
interfaces and configurations. Is there anything that can still be done to make HA community and the
general open source and Debian community closer together? HA devs are not doing anything wrong: they are
using the best open source can provide, they bring it to people whould could not install and use it otherwise,
they are contributing fixes and improvements as well. But there must be some way to do this better, together.
So I installed MariaDB, create a user and database for Firefly. I installed Firefly III and configured
it to use the MariaDB with the web config UI. When I went into the Firefly III web UI I was confronted with
the normal wizard to setup a new instance. And no reference to any backup restore. Hmm, ok. Maybe that goes
via the Importer? So I make an access token again, configured the Importer to use that, configured the
Nordlinger bank connection settings. Then I tried to import the export that I downloaded from Firefly III
before. The importer did not auto-recognose the format. Turns out it is just a list of transactions ...
It can only be barely useful if you first manually create all the asset accounts with the same names as
before and even then you'll again have to deal with resolving the problem of transfers showing up twice.
And all of your categories (that have not been used yet) are gone, your automation rules and bills are gone,
your budgets and piggy banks are gone. Boooo. It will be easier for me to recreate my account data from
bank exports again than to resolve data in that transaction export.
Turns out that Firefly III documenation explicitly
recommends making a mysqldump of your own and not rely
on anything in the app itself for backup purposes. Kind of sad this was not mentioned in the export page
that sure looked a lot like a backup :D
After doing all that work all over again I needed to make something new not to feel like I wasted days of
work for no real gain. So I started solving a problem I had for a while already - how do I add cash
transations to the system when I am out of the house with just my phone in the hand? So far my workaround has
been just sending myself messages in WhatsApp with the amount and description of any cash expenses. Two
solutions are possible: app and bot.
There are actually multiple Android-based phone apps that work with Firefly III API to do full
financial management from the phone. However, after trying it out, that is not what I will be using most
of the time. First of all this requires your Firefly III instance to be accessible from the Internet. Either
via direct API access using some port forwarding and secured with HTTPS and good access tokens, or via
a VPN server redirect that is installed on both HA and your phone. Tailscale was really easy to get working.
But the power has its drawbacks - adding a new cash transaction requires opening the app, choosing new
transaction view, entering descriptio, amount, choosing "Cash" as source account and optionally choosing
destination expense account, choosing category and budget and then submitting the form to the server.
Sadly none of that really works if you have no Internet or bad Internet at the place where you are using
cash. And it's just too many steps. Annoying.
An easier alternative is setting up a Telegram bot -
it is running in a custom Docker container right
next to your Firefly (via Portainer) and you talk to it via a custom Telegram chat channel that you create
very easily and quickly. And then you can just tell it "Coffee 5" and it will create a transaction
from the (default) cash account in 5 amount with description "Coffee". This part also works if you are
offline at the moment - the bot will receive the message once you get back online. You can use Telegram
bot menu system to edit the transaction to add categories or expense accounts, but this part only work
if you are online. And the Firefly instance does not have to be online at all. Really nifty.
So next week I will need to write up all the regular payments as bills in Firefly (again) and then I can start
writing a Python script to predict my (financial) future!region=CH means that date and number formats are also changed
to CH version, e.g. one thousand + is displayed as 1'000,50.en_US locale, with
the above number shown/formatted as 1,000.50; well, it s more
about parsing than formatting, but that s irrelevant.% defaults read .GlobalPreferences grep en_
AKLastLocale = "en_CH";
AppleLocale = "en_CH";
% defaults read -app FooBar
(has no AppleLocale key)
% defaults write -app FooBar AppleLocale en_USdefaults man page says the global-global is
NSGlobalDomain, I don t know where I got the
.GlobalPreferences. But I only needed to know the key name (in this
case, AppleLocale - of course it couldn t be LC_ALL/LANG).
One day I ll know MacOS better, but I try to learn more for 2+ years
now, and it s not a smooth ride. Old dog new tricks, right?
| Series: | Fall Revolution #3 |
| Publisher: | Tor |
| Copyright: | 1998 |
| Printing: | August 2000 |
| ISBN: | 0-8125-6858-3 |
| Format: | Mass market |
| Pages: | 305 |
Life is a process of breaking down and using other matter, and if need be, other life. Therefore, life is aggression, and successful life is successful aggression. Life is the scum of matter, and people are the scum of life. There is nothing but matter, forces, space and time, which together make power. Nothing matters, except what matters to you. Might makes right, and power makes freedom. You are free to do whatever is in your power, and if you want to survive and thrive you had better do whatever is in your interests. If your interests conflict with those of others, let the others pit their power against yours, everyone for theirselves. If your interests coincide with those of others, let them work together with you, and against the rest. We are what we eat, and we eat everything. All that you really value, and the goodness and truth and beauty of life, have their roots in this apparently barren soil. This is the true knowledge. We had founded our idealism on the most nihilistic implications of science, our socialism on crass self-interest, our peace on our capacity for mutual destruction, and our liberty on determinism. We had replaced morality with convention, bravery with safety, frugality with plenty, philosophy with science, stoicism with anaesthetics and piety with immortality. The universal acid of the true knowledge had burned away a world of words, and exposed a universe of things. Things we could use.This is certainly something that some people will believe, particularly cynical college students who love political theory, feeling smarter than other people, and calling their pet theories things like "the true knowledge." It is not even remotely believable as the governing philosophy of a solar confederation. The point of government for the average person in human society is to create and enforce predictable mutual rules that one can use as a basis for planning and habits, allowing you to not think about politics all the time. People who adore thinking about politics have great difficulty understanding how important it is to everyone else to have ignorable government. Constantly testing your power against other coalitions is a sport, not a governing philosophy. Given the implication that this testing is through violence or the threat of violence, it beggars belief that any large number of people would tolerate that type of instability for an extended period of time. Ellen is fully committed to the true knowledge. MacLeod likely is not; I don't think this represents the philosophy of the author. But the primary political conflict in this novel famous for being political science fiction is between the above variation of anarchy and an anarchocapitalist society, neither of which are believable as stable political systems for large numbers of people. This is a bit like seeking out a series because you were told it was about a great clash of European monarchies and discovering it was about a fight between Liberland and Sealand. It becomes hard to take the rest of the book seriously. I do realize that one point of political science fiction is to play with strange political ideas, similar to how science fiction plays with often-implausible science ideas. But those ideas need some contact with human nature. If you're going to tell me that the key to clawing society back from a world-wide catastrophic descent into chaos is to discard literally every social system used to create predictability and order, you had better be describing aliens, because that's not how humans work. The rest of the book is better. I am untangling a lot of backstory for the above synopsis, which in the book comes in dribs and drabs, but piecing that together is good fun. The plot is far more straightforward than the previous two books in the series: there is a clear enemy, a clear goal, and Ellen goes from point A to point B in a comprehensible way with enough twists to keep it interesting. The core moral conflict of the book is that Ellen is an anti-AI fanatic to the point that she considers anyone other than non-uploaded humans to be an existential threat. MacLeod gives the reader both reasons to believe Ellen is right and reasons to believe she's wrong, which maintains an interesting moral tension. One thing that MacLeod is very good at is what Bob Shaw called "wee thinky bits." I think my favorite in this book is the computer technology used by the Cassini Division, who have spent a century in close combat with inimical AI capable of infecting any digital computer system with tailored viruses. As a result, their computers are mechanical non-Von-Neumann machines, but mechanical with all the technology of a highly-advanced 24th century civilization with nanometer-scale manufacturing technology. It's a great mental image and a lot of fun to think about. This is the only science fiction novel that I can think of that has a hard-takeoff singularity that nonetheless is successfully resisted and fought to a stand-still by unmodified humanity. Most writers who were interested in the singularity idea treated it as either a near-total transformation leaving only remnants or as something that had to be stopped before it started. MacLeod realizes that there's no reason to believe a post-singularity form of life would be either uniform in intent or free from its own baffling sudden collapses and reversals, which can be exploited by humans. It makes for a much better story. The sociology of this book is difficult to swallow, but the characterization is significantly better than the previous books of the series and the plot is much tighter. I was too annoyed by the political science to fully enjoy it, but that may be partly the fault of my expectations coming in. If you like chewy, idea-filled science fiction with a lot of unexplained world-building that you have to puzzle out as you go, you may enjoy this, although unfortunately I think you need to read at least The Stone Canal first. The ending was a bit unsatisfying, but even that includes some neat science fiction ideas. Followed by The Sky Road, although I understand it is not a straightforward sequel. Rating: 6 out of 10
KDE Mascot
Like each month, have a look at the work funded by Freexian s Debian LTS offering.
ubuntu-release-upgrader changes your sources.list, and applies various
quirks to the upgrade.
In this post, I want to look not at the quirk aspects but discuss how dependency
solving should differ between intra-release and inter-release upgrades.
Previous solver projects (such as Mancoosi) operated under the assumption that minimizing
the number of changes performed should ultimately be the main goal of a solver. This makes
sense as every change causes risks.
However it ignores a different risk, which especially applies when upgrading from one
distribution release to a newer one: Increasing divergence from the norm.
Consider a person installs foo in Debian 12. foo depends on a b, so a will
be automatically installed to satisfy the dependency. A release later, a has some
known issues and b is prefered, the dependency now reads: b a.
A classic solver would continue to keep a installed because it was installed before,
leading upgraded installs to have foo, a installed whereas new systems have foo, b
installed. As systems get upgraded over and over, they continue to diverge further and
further from new installs to the point that it adds substantial support effort.
My proposal for the new APT solver is that when we perform release upgrades, we forget
which packages where previously automatically installed.
We effectively perform a normalization: All systems with the same set of manually installed packages will end up with the same set of automatically installed packages.
Consider the solving starting with an empty set and then installing the latest version of each previously manually installed package: It will see now that foo depends
b a and install b (and a will be removed later on as its not part of the solution).
Another case of divergence is Suggests handling. Consider that foo also Suggests
s. You now install another package bar that depends s, hence s gets installed.
Upon removing bar, s is not being removed automatically because foo still suggests
it (and you may have grown used to foo s integration of s). This is because apt considers
Suggests to be important - they won t be automatically installed, but will not be automatically
removed.
In Ubuntu, we unset that policy on release upgrades to normalize the systems. The reasoning
for that is simple: While you may have grown to use s as part of foo during the release,
an upgrade to the next release already is big enough that removing s is going to have
less of an impact - breakage of workflows is expected between release upgrades.
I believe that apt release-upgrade will benefit from both of these design choices,
and in the end it boils down to a simple mantra:
| Series: | Chilling Effect #1 |
| Publisher: | Harper Voyager |
| Copyright: | September 2019 |
| Printing: | 2020 |
| ISBN: | 0-06-287724-0 |
| Format: | Kindle |
| Pages: | 420 |
sdX in the example.
$ sudo parted --script /dev/sdX mklabel msdos $ sudo parted --script /dev/sdX mkpart primary fat32 0% 100% $ sudo mkfs.vfat /dev/sdX1 $ sudo mount /dev/sdX1 /mnt/data/
debian-testing-amd64-netinst.iso here.
$ sudo kpartx -v -a debian-testing-amd64-netinst.iso # Mount the first partition on the ISO and copy its contents to the stick $ sudo mount /dev/mapper/loop0p1 /mnt/cdrom/ $ sudo rsync -av /mnt/cdrom/ /mnt/data/ $ sudo umount /mnt/cdrom # Same story with the second partition on the ISO $ sudo mount /dev/mapper/loop0p2 /mnt/cdrom/ $ sudo rsync -av /mnt/cdrom/ /mnt/data/ $ sudo umount /mnt/cdrom $ sudo kpartx -d debian-testing-amd64-netinst.iso $ sudo umount /mnt/data
/EFI/boot/bootx64.efi,
while GRUB is at /EFI/boot/grubx64.efi. This means that if you want to test a
different shim / GRUB version, you just replace the relevant files. That s it.
Take for example /usr/lib/grub/x86_64-efi/monolithic/grubx64.efi from the
package grub-efi-amd64-bin, or the signed version from
grub-efi-amd64-signed and copy them under /EFI/boot/grubx64.efi. Or perhaps
you want to try out systemd-boot? Then take
/usr/lib/systemd/boot/efi/systemd-bootx64.efi from the package
systemd-boot-efi, copy it to /EFI/boot/bootx64.efi and you re good to go.
Figuring out the right systemd-boot configuration needed to start the Installer
is left as an exercise.
/boot/grub/grub.cfg you can pass arbitrary arguments to the kernel
and the Installer itself. See the official
Installation Guide for a comprehensive list of boot parameters.
preseed/file=/cdrom/preseed.cfg and create a /preseed.cfg file on the USB
stick. As a little example:
d-i time/zone select Europe/Rome d-i passwd/root-password this-is-the-root-password d-i passwd/root-password-again this-is-the-root-password d-i passwd/user-fullname string Emanuele Rocca d-i passwd/username string ema d-i passwd/user-password password lol-haha-uh d-i passwd/user-password-again password lol-haha-uh d-i apt-setup/no_mirror boolean true d-i popularity-contest/participate boolean true tasksel tasksel/first multiselect standard
early_command and late_command. They can be
used to execute arbitrary commands and provide thus extreme flexibility! You
can go as far as replacing parts of the installer with a sed command, or maybe
wgetting an entirely different file. This is a fairly easy way to test minor
Installer patches. As an example, I ve once used this to test a patch to
grub-installer:
d-i partman/early_command string wget https://people.debian.org/~ema/grub-installer-1035085-1 -O /usr/bin/grub-installer
$ mkdir /tmp/new-initrd $ cd /tmp/new-initrd $ zstdcat /mnt/data/install.a64/initrd.gz sudo cpio -id $ vi lib/udev/rules.d/60-block.rules $ find . cpio -o -H newc zstd --stdout > /mnt/data/install.a64/initrd.gz
early_command to replace one of
the scripts used by the Installer, namely grub-installer. It turns out that
such script is installed by a udeb, so let s do things right and build a new
Installer ISO with our custom grub udeb.
dpkg-buildpackage -rfakeroot.
$ git clone https://salsa.debian.org/installer-team/debian-installer/ $ cd debian-installer/ $ sudo apt build-dep .
grub-installer udeb to the localudebs directory and create a
new netboot image:
$ cp /path/to/grub-installer_1.198_arm64.udeb build/localudebs/ $ cd build $ fakeroot make clean_netboot build_netboot
dest/netboot/mini.iso.
$ git clone https://salsa.debian.org/kernel-team/linux/ $ ./debian/bin/genorig.py https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
linux-stable.
debian/config/amd64/config. Don t worry about where you put it in the file,
there s a tool from https://salsa.debian.org/kernel-team/kernel-team to fix that:
$ /path/to/kernel-team/utils/kconfigeditor2/process.py .
$ export MAKEFLAGS=-j$(nproc) $ export DEB_BUILD_PROFILES='pkg.linux.nokerneldbg pkg.linux.nokerneldbginfo pkg.linux.notools nodoc' $ debian/rules orig $ debian/rules debian/control $ dpkg-buildpackage -b -nc -uc
linux_6.6~rc3-1~exp1_arm64.changes here. To
generate the udebs used by the Installer you need to first get a linux-signed
.dsc file, and then build it with sbuild in this example:
$ /path/to/kernel-team/scripts/debian-test-sign linux_6.6~rc3-1~exp1_arm64.changes $ sbuild --dist=unstable --extra-package=$PWD linux-signed-arm64_6.6~rc3+1~exp1.dsc
debian-installer/build/localudebs/ and then run fakeroot make clean_netboot
build_netboot as described in the previous section. In case you are trying to
use a different kernel version from what is currently in sid, you will have to
install the linux-image package on the system building the ISO, and change
LINUX_KERNEL_ABI in build/config/common. The linux-image dependency in
debian/control probably needs to be tweaked as well.
anna will
complain that
your new kernel cannot be found in the archive. Copy the kernel udebs you have
built onto a vfat formatted USB stick, switch to a terminal, and install them
all with udpkg:
~ # udpkg -i *.udeb
| Publisher: | Amazon Original Stories |
| Copyright: | June 2023 |
| ISBN: | 1-6625-1572-3 |
| ISBN: | 1-6625-1622-3 |
| ISBN: | 1-6625-1503-0 |
| ISBN: | 1-6625-1567-7 |
| ISBN: | 1-6625-1678-9 |
| ISBN: | 1-6625-1533-2 |
| Format: | Kindle |
| Pages: | 219 |
Today, when going through a box of old T-shirts, I found the shirt I was looking for to bring to the occasion: [...] For the benefit of people who read this using a non-image-displaying browser or RSS client, they are respectively:Now I don't know about you, but I really expected someone to come up with an answer to this, directly on Debian Planet! I have patiently waited for such an answer but enough is enough, I'm a Debian member, surely I can cull all of this together. So, low and behold, here are the actual numbers from 2023!and10 years 100 countries 1000 maintainers 10000 packages20 years ago we celebrated eating grilled meat at J0rd1 s house. This year, we had vegan tostadas in the menu. And maybe we are no longer that young, but we are still very proud and happy of our project! Now How would numbers line up today for Debian, 20 years later? Have we managed to get the bugs fixed line increase by a factor of 10? Quite probably, the lines of code we also have, and I can only guess the number of users and installations, which was already just a wild guess back then, might have multiplied by over 10, at least if we count indirect users and installs as well1 project 10 architectures 100 countries 1000 maintainers 10000 packages 100000 bugs fixed 1000000 installations 10000000 users 100000000 lines of code
i386, m68k, Alpha, SPARC, PowerPC, ARM, IA-64, hppa, mips,
s390; while bookworm released with actually 9 supported
architectures instead of 10: i386, amd64, aarch64, armel,
armhf, mipsel, mips64el, ppc64el, s390x)ldapsearch -b
ou=users,dc=debian,dc=org -D uid=anarcat,ou=users,dc=debian,dc=org
-ZZ -vLxW '(c=*)' c grep ^c: sort uniq -c sort -n wc -l
on coccia)SELECT COUNT(DISTINCT package) FROM all_packages; => 211151)SELECT COUNT(id),status FROM all_bugs GROUP BY status;) 1 project
10 architectures
30 years
100 countries (actually 63, but we'd like to have yours!)
1000 maintainers (yep, still there!)
35000 packages
211000 *binary* packages
1000000 bugs fixed
1000000000 lines of code
uncounted installations and users, we don't track you
1 project
10 architectures
100 countries (actually 63, but we'd like to have yours!)
1000 maintainers (yep, still there!)
100000 packages
1000000 bugs fixed
1000000000 lines of code
uncounted installations and users, we don't track you
100 years in there, but that would be
rounding up too much. Let's give it another 30 years first.
Hopefully, some real scientist is going to balk at this crude
methodology and come up with some more interesting numbers for the
next t-shirt. Otherwise I'm available for bar mitzvahs and children
parties.
I very, very nearly didn t make it to DebConf this year, I had a bad cold/flu for a few days before I left, and after a negative covid-19 test just minutes before my flight, I decided to take the plunge and travel.
This is just everything in chronological order, more or less, it s the only way I could write it.
If you got one of these Cheese & Wine bags from DebConf, that s from the South African local group!
Some hopefully harmless soldering.
git secret_cabal greetmr) to manage multiple git repositories with one command;/etc directory;.mr directory is where everything is managed: I don t have a single .mrconfig file but a few different ones, that in turn load all files in a directory with the same name:
collections.mr for the media file annexes and inventories (split into different files, so that computers with little disk space can only get the inventories);private.mr for stuff that should only go on my own personal machine, not on shared ones;projects.mr for the actual projects, with different files for the kinds of projects (software, docs, packaging, crafts, etc.);setup.mr with all of the vcsh repositories, including the one that tracks the mr files (I ll talk about the circular dependency later);work.mr for repositories that are related to $DAYJOB..mr/machines directory, each one of which has the list of repositories that should be on every specific machine, including a generic workstation, but also specific machines such as e.g. the media center which has a custom set of repositories.
The dot files from my home directory are kept in vcsh, so that it s easy to split them out into different repositories, and I m mostly used the simplest configuration described in the 30 Second How-to in its homepage; vcsh gives some commands to work on all vcsh repositories at the same time, but most of the time I work on a single repository, and use mr to act on more than one repo.
The media collections are also pretty straightforward git-annex repositories, one for each kind of media (music, movies and other videos, e-books, pictures, etc.) and I don t use any auto-syncing features but simply copy and move files around between clones with the git annex copy, git annex move and git annex get commands.
There isn t much to say about the project repositories (plain git), and I think that the way I use my own program lesana for inventories and project tracking is worth an article of its own, here I ll just say that the file format used has been designed (of course) to work nicely with git.
On every machine I install etckeeper so that there is a history of the changes in the /etc directory, but that s only a local repository, not stored anywhere else, and is used mostly in case something breaks with an update or in similar situation. The authoritative source for the configuration of each machine is an ansible playbook (of course saved in git) which can be used to fully reconfigure the machine from a bare Debian installation.
When such a reconfiguration from scratch happens, it will be in two stages: first a run of ansible does the system-wide configuration (including installing packages, creating users etc.), and then I login on the machine and run mr to set up my own home. Of course there is a chicken-and-egg problem in that I need the mr configuration to know where to get the mr configuration, and that is solved by having setup two vcsh repositories from an old tarball export: the one with the ssh configuration to access the repositories and the one with the mr files.
So, after a machine has been configured with ansible what I ll actually do is to login, use vcsh pull to update those two repositories and then run mr to checkout everything else.
And that s it, if you have questions on something feel free to ask me on the fediverse or via email (contacts are in the about page)
Update (2023-09-12 17:00ish): The ~/.mr directory is not special for mr, it s just what I use and then I always run mr -c ~/.mr/some/suitable/file.mr, with the actual file being different whether I m registering a new repo or checking out / updating them. I could include some appropriate ~/.mr/machines/some_machine.mr in ~/.mrconfig, but I ve never bothered to do so, since it wouldn t cover all usecases anyway. Thanks to the person on #vcs-home@OFTC who asked me the question :)
Like each month, have a look at the work funded by Freexian s Debian LTS offering.
Next.